Skip to content

spec(FIX-1438): a run that stopped at its budget still closes its row as done - #1887

Closed
claude[bot] wants to merge 4 commits into
mainfrom
spec/FIX-1438
Closed

claude[bot] wants to merge 4 commits into
mainfrom
spec/FIX-1438

Conversation

@claude

@claude claude Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Jake · project thread

Someone who… Today After
watches a board to see what got finished A row reads completed for a run that ran out of turns half-way and committed the half it managed. Nothing says so The row comes back open with the reason on it, and the work carries on from where the run stopped
writes a phase's completion check Gets one fact — is the job done on the branch. The run's own report that it ran out of road never reaches them Gets both facts: the state of the branch, and the word the run used for how it stopped. The call stays theirs
runs a coding agent on a tight turn or spend budget Budget stops are indistinguishable from clean finishes once a row settles A budget stop is visible on the row, every time it happens
has a run that stopped at its budget but had genuinely finished The row closes, correctly, by luck The row re-opens once. The next attempt resumes the same coding session, finds the work done, and closes it. One extra cheap run
reads the harness-manager contract to build a phase The contract says a clean end plus a passing check is completion, and never says that "clean" includes a run that hit its limit The contract says which two facts decide completion, and that ignoring the second one closes rows on partial work

Two columns. Today, a finished run reports both a status and an outcome word, only the status reaches the completion check, the outcome word goes nowhere, and the row settles completed. After, both reach the check, the check answers not done, and the row re-opens with the reason attached.

Follow the second line out of each box. Today it dead-ends: the run's own word for how it stopped is recorded and no decision reads it, so a half-finished commit closes the row.

How: a supervised run reports two facts when it ends, and only one is read. The second one is handed to the phase's completion check, which stays the only thing that decides. The manager compares nothing on it.

Sign off

  1. The phase's completion check receives the run's outcome word; the manager still decides nothing on it. If wrong: every phase carries one more call the framework cannot enforce, so a phase that forgets closes rows on partial work exactly as today, silently.
  2. Both shipped phases refuse a budget stop — including the one whose check is "a pull request exists". If wrong: a run that opened a reviewable pull request and then hit its limit costs one extra run before it closes.
  3. A refused budget stop re-opens the row with its reason attached, and errors once the retry budget is spent. If wrong: a run that keeps stopping at its budget burns its whole retry budget before anyone is asked about it.

Open: none. Number 1 is the one to weigh — it is the choice to hand the fact down rather than rule on it centrally. Reasoning and what lost: DECISIONS.md. The cases: BUSINESS-RULES.md.

Reviewers · look here

  • Decisions → D1. Handing the fact to each phase fixes nothing on its own; D3 is where rows actually stop closing. Is that better than one rule in the manager, which fixes every phase at once and is wrong for the phase where a budget stop really did leave the job done?
  • Decisions → D3. Conductor's check is "a pull request exists". Refusing there costs one extra run on every budget stop that had genuinely finished. Worth it?
  • Rules → BR-15. The lab's stopped-at-limit control stops being special-cased and is caught by the same assertion as the others. If that reads wrong, the removal in the plan's S6 is wrong.

Not here: any new board status. No partial, no second hold, no needs_input — locked on the issue, and a refused budget stop settles through terminals that already exist.

Spec · Decisions · Rules · Plan · FIX-1438 · under FIX-1408 (W4) · never merges

Evidence — the finding, reproduced both ways

Run on a34a3ab, this branch's base, 2026-09-18. No model, no network.

GOAL_CONTROL=stopped-at-limit pnpm tsx goals/devforce-lab/it-wakes-the-seat-a-file-declared/run.mts
# exit 1 — FAIL: the run reported outcome "stopped-at-limit" and the row settled done anyway

pnpm tsx goals/devforce-lab/it-wakes-the-seat-a-file-declared/run.mts
# exit 0 — PASS

No POC was built. The reproduction is the shipped lab, and both directions were run before the spec was written.

How to review this — altitude, what's in scope, what's deliberately unsettled

This is a direction document in four files, not an implementation. Reviewing it well means
answering one question: is this the right approach?

In scope to challenge:

  • The problem framing in SPEC.md — are we solving the right thing, for the people named?
  • The approach — will it work, does it fit the architecture and docs/philosophy.md?
  • Any numbered decision in DECISIONS.md — that's the sign-off surface.
  • A case BUSINESS-RULES.md misses, or a rule that would invalidate the design.
  • Scope — a deliverable that shouldn't ship, or one that's missing.

Out of scope — deliberately unsettled here, and owned by the implementing agent:

  • Names, signatures, file paths, and module layout beyond the few PLAN.md pins on purpose.
  • Local structure: which helper, how a function is decomposed, error-message wording.
  • Micro-optimizations and style preferences.
  • Test names and internal test structure (the behaviours are the rules; the tests are not).
  • Anything PLAN.md left open on purpose — it is directional by design, and a gap at that
    altitude is intended, not an omission.
  • The figures, at the pixel level. A figure is right when it reads right. A misplaced label
    or a colour you'd have chosen differently is not a finding; a figure that shows a shape the
    decisions don't say is.
  • The solution sketch, at the line level. PLAN.md may include a rough pseudocode sketch
    showing the shape of the proposed solution. It is knowingly incomplete, is not real code,
    and is not what will ship. Review it for directional viability only. Do not report
    that it lacks error handling, omits edge cases, names things that don't exist in the repo, or
    wouldn't build: all of that is true on purpose.
  • Any POC files on this branch, entirely. This PR is never merged, so it may carry throwaway
    proof-of-concept code, characterization tests, or an HTML mockup built to validate the
    direction (spec-poc). None of it ships. Read it to
    judge whether the shape holds; do not review it as code.

One request, if you are an automated reviewer: the two lists above are the whole difference
between a useful review of this document and a long one. Volume is not signal here — a single
finding above the line is worth more than twenty below it.

Feedback in the second list is welcome and gets recorded verbatim in PLAN.md → Notes from review for the implementer to weigh against real code. It will not be argued with, and it
will not be folded into the design prose. Please don't re-raise it: one mention is enough.

The one exception is the sketch and the POC. Line-level feedback on either is dropped, not
recorded — there is nothing for an implementer to weigh about code that isn't shipping. Feedback
about their direction is not line-level and is treated like any other above-the-bar finding.

🤖 Generated with Claude Code

https://claude.ai/code/session_018VJZRfkY7gBbiYZhr1eanR


Generated by Claude Code

RetriggerConfidence Score: 4/5

The proposed direction appears safe to proceed with after correcting the non-blocking outcome-persistence ambiguity in the problem statement.

Fix All in CursorFindings

  1. P2 Outcome fields are conflated ▶

Summary

This direction document proposes passing the harness-reported terminal outcome into each phase’s completion check while leaving completion authority with the phase.

  • Both shipped phases will reject stopped-at-limit before applying their branch or pull-request probes.
  • A rejection follows the existing retry path, preserving feedback and the confirmed coding session.
  • The plan covers forward-compatible and absent outcome values, question precedence, documentation, release notes, and the DevForce regression goal.
  • The spec should clearly distinguish the transient harness outcome from the run record’s separate bookkeeping outcome.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  H["Harness result"] --> S["status"]
  H --> O["reported outcome"]
  S --> M["Manager verdict handler"]
  O --> M
  M -->|"question present"| Q["Park row"]
  M -->|"clean run"| P["Phase completion check"]
  P -->|"done"| D["Settle row"]
  P -->|"not done"| R["Record reason and retry"]
  R -->|"budget remains"| N["Re-open and resume session"]
  R -->|"budget exhausted"| E["Error row"]
Loading

Reviews (1) · Last reviewed commit: "spec(FIX-1438): a run that stopped at it..."

… as done

The phase's completion check receives the run's outcome word; the manager
still decides nothing on it. Both shipped phases refuse a budget stop, and
a refusal settles through the re-pend path that already exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018VJZRfkY7gBbiYZhr1eanR
@vercel

vercel Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Sep 18, 2026 2:05am UTC
flow-state-dev-kitchen-sink Ignored Ignored Sep 18, 2026 2:05am UTC
flow-state-dev-ui Ignored Ignored Preview Sep 18, 2026 2:05am UTC

Request Review

@claude claude Bot added the spec label Sep 18, 2026
@linear-code

linear-code Bot commented Sep 18, 2026

Copy link
Copy Markdown

FIX-1438

@claude
claude Bot requested a review from jhoffner September 18, 2026 01:53
@jhoffner

Copy link
Copy Markdown
Collaborator

FSD Architect — coherence check only. Not owner approval, not the merge gate (never-merge spec; shared jhoffner login).

No two-direction fork. This settles the FIX-1438 enrich open wall as option 2 (phase receives the harness how-it-stopped word; manager does not override the done-condition), refuses new TaskStatus / second hold / needs_input (D-1 / FIX-1245 fence), and soft-parallels FIX-1440 / #1888 without blocking. Complementary to LAB-162 ask→park (BR-12), not a second human-wait model. Jake-requested — treating D1–D3 as owner-settled on product meaning of completed; not a D-n for eng.

One clarity ask (what to change): in BUSINESS-RULES.md → Failure taxonomy, the line that the run record's outcome keeps running / succeeded / failed collides with the harness contract's HarnessRunOutcome (finished | stopped-at-limit | failed) that SPEC/D1/D3 and packages/core already teach. Say which field you mean (handle vs any separate bookkeeping), or drop that sentence so implementers do not invent a second enum. The rest of the rules already use the harness word correctly.

Soft (not blocking): product name is DevForce (D-12); labs/conductor as the package path is fine — prefer DevForce when teaching the Lab in new prose.

No Jake ping.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify review (spec only)

Verdict: The chosen direction is the smallest correct fix for the constraints you already locked (no manager authority inversion, no new board status, phase owns "done"). Implementation should mirror what you already have for prompts: a CompletionRunContext sibling of PromptRunContext, pass handle.outcome only in decide arm 2, and keep the manager from comparing the word except when phrasing a refusal (S3). That is one field + one spread + two one-line phase guards + lab de-special-case — not a new subsystem.

What reads heavier than the code is mostly spec surface area, not the architecture.

Where the approach did not overreach

  • D1 (hand to phase, don't enforce) is expensive in prose because the cost is real (BR-7 / silent partial close). Re-opening a manager default with opt-out would shrink implementation but trades away the issue's authority model; the spec argues that honestly in D1 and Follow-ups.
  • D3 (both phases refuse, including PR probe) is the actual behavior change; D1 alone fixes nothing, and the spec says so plainly.
  • Rejected paths (document-only, manager veto, new TaskStatus, boolean collapse of unknown/absent) are the simpler forks; dropping them is correct for FIX-1438's goal.

Simpler spec shape (optional polish, not a fork)

These would cut repetition without changing sign-off:

  1. One glossary line for two different "outcome" meanings (harness stop report: finished / stopped-at-limit / failed vs run-row bookkeeping: running / succeeded / failed). Right now SPEC and Failure taxonomy use the same word for different enums — easy implementer footgun.
  2. Single owner per invariant: prompt/completion split → BR-6 once; pass-through word → one BR + one PLAN guardrail; D1's "silent if ignored" cost → DECISIONS D1 only (SPEC sign-off #1 can stay one line).
  3. Thin PLAN surfaces: rewrite S1 as "type isDone receives only" (existing PromptRunContext pattern), merge S4+S5 into one row with two paths, merge S6+S7, fold S3 into S2; drop or fold V8 into VG.
  4. Tests: table-drive the four outcome values once for V1+V2; keep V7 on injected probes — avoid duplicating eight manager scenarios or live gh in V6.

Performance (product, not CPU)

Dominant cost is extra full agent attempts when D3 refuses a budget stop (especially conductor). Early refuse before PR/git probes saves I/O on the failure path but does not offset retries. The spec names that trade in D3; no framework hot-path concern in S2.

Alternative I would not take instead

  • Manager-level "never settle on stopped-at-limit" — fixes every phase at once but blocks the "budget hit limit but job actually done" path you describe in the five-people table.
  • Persist harness stop on the run row and read it inside isDone — extra persistence + blurs who decides completion; today's row outcome is attempt bookkeeping, not the harness three-way word.
  • Third-party package — nothing here; this is your harness contract seam.

Approval: Direction and plan sequence are right for a 1 PR implementation. I'd still fold the glossary + doc dedupe before or during implement so the worker doesn't re-derive four copies of the same guardrail.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread spec/FIX-1438/SPEC.md Resolved
Comment thread spec/FIX-1438/PLAN.md Resolved
Comment thread spec/FIX-1438/PLAN.md Resolved
Comment thread spec/FIX-1438/BUSINESS-RULES.md Resolved
Comment thread spec/FIX-1438/DECISIONS.md Resolved
Comment thread spec/FIX-1438/SPEC.md Resolved
…s outcome

Re-draft: the draft repeated the issue's claim that the harness's
stop report is written to the run record. It is not — it reaches no
store, and is read only by the text of a failure message. The two
three-value fields are now named apart in all four documents and the
figure, and the plan pins the phase-side field away from `outcome`.
No decision moved.

Re-draft: review round 1

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018VJZRfkY7gBbiYZhr1eanR
@claude

claude Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Round 1 folded — 3eb7ef9. Noted that this is a coherence check, not the owner's approval; the gate is still open.

Your clarity ask, and it was more than a collision. Three reviewers landed on the same line, so I checked it against the code rather than just renaming. handle.outcome appears exactly once in packages/harness-manager/src/manager.ts — the failure-message text in the verdict handler's third arm. No writeRunRow call carries it, and the run row's outcome is only ever written running, succeeded or failed. So the draft's claim that the stop report is "written to the run record and never consulted" is false: it is written nowhere, and it reaches the row only as text inside reason on a failed attempt.

That sentence came from FIX-1438's own Problem section, which says the same thing. The spec is corrected; the Linear issue still carries the original claim — I have not rewritten someone else's stated evidence without flagging it. Say the word and I will patch that sentence on the issue too.

What changed:

  • Stop report (finished / stopped-at-limit / failed, on the handle) and row outcome (running / succeeded / failed, on the run record) are named apart in all four documents and in the figure. SPEC.md defines them once; the others point at it.
  • PLAN.md pins one more name: the field a phase reads the stop report from is not spelled outcome. Two fields one word apart is how the wrong one gets wired.
  • A guardrail: the run record's outcome is untouched, and nothing writes the stop report to it.
  • DECISIONS.md → How it got here carries the correction. No decision moved — D1, D2 and D3 are unchanged.

On the soft note: agreed, and the spec already teaches the Lab as DevForce throughout. labs/conductor and "conductor's implement phase" appear only where a package path or a shipped phase is being named. I added a line to the docs plan so the same holds in the published prose.

Not folded, recorded as implementer notes (PLAN.md → Notes from review): the surface merges (S4+S5, S6+S7, S3 into S2, V8 into VG), the shared four-way fixture, injected probes for V7 and V6, and the single-owner-per-invariant trim. S1's wording is rephrased to stop reading like a new type hierarchy, and V1 now says one table covers it and V2. The four documents are written for four readers, so a decision card and a plan guardrail saying related things is the template's shape rather than repetition — that one stays as it is.

Round 1 of the two-round budget.


Generated by Claude Code

Round 1 added a third pinned name (the field a phase reads the stop
report from must not be plainly `outcome`) without updating the
heading above the table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018VJZRfkY7gBbiYZhr1eanR
…ow-ups

Two lines still said "the outcome" where they mean the run's stop
report: the README docs bullet and the follow-up naming D1's cost.
Every remaining use of "outcome" in the set now refers to the run
record's own field, or quotes the lab's assertion text verbatim.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018VJZRfkY7gBbiYZhr1eanR
@jhoffner jhoffner added the spec approved This spec is ready for implementation label Sep 18, 2026
@jhoffner

Copy link
Copy Markdown
Collaborator

Approved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spec approved This spec is ready for implementation spec

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants